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ABSTRACT 

While the logic of science serves as the 
methodological framework for psychological research, its particular 
merit for educational evaluation has yet to filter into pract:icalj 
applica1:ion« The logic of science provides a strategy for evaluating 
large and complex educational treatments that is a useful guide to 
program development and improvement. This paper explicates a method 
of scientific inquiry applicable to the formative evaluation of 
educational progirams and, in so doing, illuminates an artificial 
distinction between the ways in which concepts are validated in 
science and education. (Author) 
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HYPOTHETICAL CONCEPTS, INTERVENING CONSTRUCTS, 
AND OBSERVED DATA IN PROGRAM EV. //JATION 
Gary D. Borich 
Research and Development Center for Teacher Education 
The University of Texas at Austin 

While the logic of science serves as the methodological framework 
for psychological research, its particular merit for program development 
and evaluation has yet to filter into practical application. The logic 
of science offers a basic strategy that can be used as a comprehensive 
guide to the formative evaluation o£ large and complex educational programs. 
The purpose of this paper is to explicate the logic and method of scientific 
inquiry traditional to psychological research, but applicable to the develop- 
ment and evaluation of educational programn. 

The logic of science depicts educational programs as a network of 
instructional concepts, intervening constructs, and observable data so 
that a hierarchy from perceptual (observable) to conceptual, subordinate 
to superordinate, specific 'to abstract is built with the most specific and 
reliable data subordinate and the least specific and least reliable 
concepts superordinate. The continuum is basic to scientific inquiry 
whether in the physical sciences, hiraianities or education. The researcher 
studies observables as basic datum from which he builds higher order con- 
structs and concepts. Figure 1 illustrates the organization of a mathematics 
curriculum which mirrors the scientific mode of inquiry upon which it is 
based • 
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In a well-defined curriculum, movement from the perceptual to the 
conceptual is well mapped with passages between concepts short and reliable 
so as to create the maximum probability of the proposition, if A then B. 
The reliability of a passage is expressed as the ratio of pupils who 
achieve learning set A to those who obtain the superordinate set, B. 
For curricula that are well developed even the most hierarchical of concepts 
may be closely tied to observables through several intermediate concepts 
which differ from observables in only small degrees of abstraction. 
Curricula patterned after Gagne's work (1965) serve as practical examples 
of the scientific model. 

Large and complex educational programs, however, are rarely so tight 
or so logical. For an educational program, passages from observables to 
the intermediate constructs may be short and -teliable, while passages from 
the intermediate to the hierarchical concepts are likely to be long and 
circuitous. Because hierarchical concepts are tenuous, their development 
and evaluation is difficult. For large, complex educational programs 
a model based upon methods of scientific inquiry may be more appropriate 
for formative program improvements than traditional evaluation schemes. 
Hie following discussion serves to explicate a proposed methodology for 
formative evaluation aftar which an example will illustrate its use in a 
large and complex educational program. 

A definition of "concept ." Although program developers and evaluators 
use the word "concept" almost daily, its use is often devoid of explicit 
meaning. Psychologists have added to the ambiguity by often using the words 
"concept" and "construct" interchangeably. 



Concepts themselves vary so that some may be so abstract as to 
elicit from memory little or no observable data^ such as the concept of 
"meaning" itself. Or they may be more Intuitive, such as the concept of 
"man" or "tree*" Or, still yet, they may be burgeoning with rational and 
empirical relationships such as the concept "number," Our i.ost hier- 
archical concepts for complex educational programs are usually intuitive 
and often coincide with the theme or purpose of a particular program such 
as "personalized education for teachers," "individualized instruction for 
the gifted," "remedial instruction for the disadvantaged," all of which 
imply underlying behaviors for teachers, the gifted, and the disadvantaged 
which are expected to change as a function of some treatment. Often the 
most hierarchical concept (personalization, individualization, remediation) 
is the only concept with which the program developer begins his work* While 
the curriculum developer works with concepts that have known relationships 
within and between substantive disciplines, the program developer faces a 
lack of specificity and a need for a viable plan which can link super- 
ordinate program concepts and constructs with observable behavioral data* 

To the developer a distinction between concept and construct is useful. 
On a continuum of superordination constructs lie between concepts and observ«* 
ables and wiien quantified for measurement purposes are referred to as inter- 
vening variables* Constructs are defined in terms of their antecedents 
which comprise observable data and are dependent on them for their meaning* 
Concepts however are inferred variables that may or may not have clear 
empirical antecedents. A concept without clear empirical antecedents is 
a hypothetical construct (HC). HC's are not reducible to that of their 



antecedents and therefore a degree of error must be expected in passing 
to the inferred concept from its hypothesized antecedents. 

The term'"superordinate" has been used by Rczeboom (1956) to denote 
a continuum of HC's which vary in degrees of uncertainty. Hie HC of a 
program is its most uncertain concept. The developer works to build into 
the program antecedents which provide clear and direct relationships to 
its superordinate concept while the evaluator focuses on the extent to 
which these relationships are in fact clear and direct. At least three 
degrees of HC uncertainty may be identified. 

Degree of HC uncertainty . The least uncertain of these is an HC for 
^ich there are specified antecedents whose prediction/of higher order 
concepts is uncertain. The HC is positioned a step above an array of more 
specific intervening constructs which could serve as antecedents when their 
relatic iship to the HC is empirically confirmed. Figure 2 depicts attitude 
as an intervening construct (IC), intermediate in superordination to 
individualized learning and observed program inputs. The HC takes on 
meaning with the specification of an IC as one of its components. Many 
such IC*s are needed to operationally define an HC. 



individualized learning 






observable program inputs 

Figure 2 Intervening construct linking an observable 

program input to a hypothetical program concept 



A second more uncertain HC is one in vhich its constituent properties 
exist separately, but whose joint existence is uncertain* The network 
becomes more complex with the insertion of additional IC^s whose relations- 
ship to HC is still uncertain* Figure 3 represents an expanded network 
with motivation and two subsidiary constructs in a hypothesized relation- 
ship to HC* 
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Figure 3 Expanded network of intervening constructs 
linking observable program inputs to a 
hypothetical program concept 



Although attitude toward instruction and motivation have been well 
documented as constructs their hypothesized joint occurrence in relation 
to individualized learning is specific to the nature of the program being 
studied* The task of supporting or refuting these constructs as components 
of individualized learning via correlation or experimental manipulation is 
a primary task for program evaluation* 

The third and most uncertain type of HC is one in which at least 
one antecedent construct assumes a theoretical relationship which as yet 



has not been established and therefore one in vhich the connection between 
construct and concept is in a highly tenuous state. Progranj developers hypo- 
thesize a general relationship neither confirmed nor refuted by current 
theory. If the hypothesized relation is critical to the HC, i.e., not 
simply an alternative explanation of it, progirn evaluation must posit $ 
theory that can account for the critical relationship. This may well be 
the case in Figure 3, hence the broken lines between con&tructs and concept. 
A program built on many such hypothetical relationships becomes a heuristic 
for research more than a program that Is expected to engender observable 
behavioral outcomes. Tenuous relationships posed as alternative hypotheses, 
however, provide important additions to the network as rival hypotheses 
for follow-up exploration. 

Passage of data to orderly knowledge The program developer either 
consciously or unconsciously posits bonds between constructs and concepts 
during the development process. Northrop (1946) has called these bonds 
^'epistemic correlations," Margeneau (1950) "passages" and Morris (1946) 
"sjmtax" vis a vis "semantics," while the hard scientist speaks of "rules" 
or "laws of correspondence." The accuracy of prediction that is connoted 
from "rule" or "law" might be less appropriate for education than for the 
hard sciences where the reliability of measurements is often sufficient to 
warrant their use. Northrop 's and Margeneau *s terms may be more helpful 
in that a developer should hypothesize links between constructs and concepts 
that connect the empirical and hypothetical components of a program. 
Several kinds of passages are possible. 

The most uncertain pass<ige is often the result of endowing a concept 
with passages to and from constructs that are logical rather than behavioral. 
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A program to personall^se the education of teachers might; be described, 
for example, with "assessment,^* "individualized Ikis true t ion," and 
"feedback" components which in themselves are descriptors that, although 
organize the program, do little to behaviorally define it. If "personalized 
teachers act differently than other teachers, the program developer needs 
to know the behaviors that are prerequisite to or correlated with the HC 
which underlies the program, e.g., effective teaching. While the program 
proposal may specify assessment, individualized instruction, and feedback 
as program components, the developer must posit tiie importance of anxiery 
ip relation to assessment, cognitive style in relation to individualized 
instruction and openness in relation to feedback « Af^er psychological 
constructs are specified, the evaluator can determire whether or not 
feedback, assessment, and individualized instruction in their proposed 
form are congruent with and in fact engender the desired terminal behavior. 
Categorical descriptors a program fail to define the behavioral under** 
pinnings of a concept and often serve as %lack" boxes which mask a plethora 
of behaviors \^ich the developer needs to identify in positing passages 
to the HC. 

A more fruitful approach to positing and defining concepts is to 
determine a sequence of behaviors that is intermediate to terminal behaviors 
After reviewing a repertoire of behaviors that are related to HC and the 
intercorrelations that might be expected among them, the program developer 
specifies that y is likely to be a function of Xj^...x^; where Xj^...x^^ 
arc observed program inputs and y an intervening variable. The evaluator 
might posit causal connections between Xj^, Xj, x^ leading to y or a hier- 
archical arrangement in which each antecedent directly relates to y as in 
Figure 4. 
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Like data are grouped under a single Intervening variable In order to 
build the most parsimonious network. Intervening constructs are related 
to observed data beneath it and hypothetically to the concepts above it. 

Program evaluation substantiates in1:ervenins constructs by confirming 
passages to and from them with correlational techniqi'C:*; • Correlc^tions and 
regression equations are calculated to define the relationship between 
observable Jata and intervening constructs. Greater than chance correla- 
tions are expected betwe<.a passages Xj^...X2 and y and x^...Xg and z. Ihese 
correlations confirm the magnitude of the relationship while a prediction 
equation can mathematically describe; the rate at which y Increases relative 
tox (e.g., for every unit increa^^e in x there is a two*unit increase in y 
80 that when x « 2, y « 4, and when x 3, y • 5). The deve!.cper is 
especially interested in relationships in ^hich a small change in x coincides 
with a large change in y. For these variables small observable clmnges are 
likely to be mort directly limced to the HC. If an intervening construct 
cannot be substantiated by any variables on the observable levels it should 
be removed from the network and from any proposed definition of HC. 



Correlations are expected of leiSs magnitude between y and HC, but if no 
relationship exists either the y construct and its observable counterparts 
must be dropped from the network or the program definition changed to 
reflect a more appropriate HC. 

Verifying hypothetical concepts . Correlations allow us to pass from 
observables to the field of constructs and when used in reverse can verify 
the existence of hypothetical concepts. Reversals can be either trivial 
or non-trivial. A trivial reversal occurs ^en the path which leu originally 
to the formation of a construct or concept is retraced as is illustrated 
with the dotted line between HC and W in Figure 5. 




Figure 5 Trivial and non-trivial reversals in a network 
of intervening constructs 

No new relationships are added to the definition of HC by the dotted line 
and HC becomes in effect the antecedent, W. A non-trivial reversal adds 
to HC by following a new path to other antecedents. HC becomes less 
tenuous as we substantiate relationships between it and other constructs. 
A relationship between HC and X is expected from a knowledge of relation- 
ships between X and W and between W and HC. The longer the path, the weaker 
the correlation is expected to be, so that X may account for only a portion 
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of the variance accounted for in HC and this must be less than that 
accounted for by W. Should the reverse occur the network is redrawn 
placing X in a superordinate position to W. The greater the distance to 
and from HC by any given loop, the more non- trivial the reversal. By 
establishing a complex of interrelationships more constructs are likely 
to share variance with HC and therefore become part of its definition. 

Constructs Y and Z suggest two other likely results. Y represents 
a peninsular construct which shares variance with X but with no other 
construct. Y has utility only with respect to a definition of X but may 
relate at some later time to additions In the network that may in turn 
relate to HC. Z on the other hand is an island construct that relates to 
no other construct. Z has potential whenever new constructs are added to 
the network, as does Y. Empirical relationships are posited among all 
constructs and concepts in the network. These empirical relationships 
account in part for changes in HC \rtiile defining and validating the 
program. 

Correlational techniques, while helpful in the validation of inter- 
vening constructs, caanot alone constitute program evaluation. At some 
point the evaluator and developer seek passages between constructs and 
concepts that are causal. The program developer needs to Ic^arn whethi^.r, 
for example, high anxious participants might better be dropped from the 
program or perhaps receive a less threatening mode of assessment, whether 
one cognitive style might be more appropriate to individualized instruction 
than another, or whether the extent to which a participant Is open to 
feedback dictates its value and efficiency. The evaluator seeks to 
specify an optimal value or permissible range of values for the intervening 
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constructs \Aich relate to terminal behaviors. While Yee and Gage (1968) 
and Duncan (1966) suggest techniques for determining causal inferences 
from correlational data well worth studying, the evaluator may also have 
the opportunity to manipulate the instructional components of a program 
if only by using random or fortuito-ts fluctuations in program implementa- 
tion. The time a program participant spends in a feedback session may, 
for example, differ as a function of the idiosyncracies of both program 
and participant. While some participants may receive brief feedback 
sessions due to tight class schedules others may receive lengthy sessions 
at more leisurely times, thereby altering the amount of feedback the program 
provides . 

Variations such as these can be documented and studied as degrees of 
program implementation and constitute observable program inputs. By 
availing himself of fortuitous comparisons the evaluator moves from the 
observables (differing amounts of feedback) to an intervening construct 
(anxiety) and is now in a position to examine the relationship between 
anxiety and the hypothesized terminal behaviors using inferential tech- 
niques. Without traditional control groups and pre/post assessments the 
evaluator can often construct a sophisticated desif n which provides formative 
data to the developer and \^ich evaluates the efficacy of moving from 
observed program inputs to hypothesized terminal behaviors through passages 
posited by the program. The evaluator begins by hypothesizing a network 
of constructs as depicted in Figure 6 and then measuring relationships 
between instructional variables and intervening constructs, and relating 
intervening constructs, in turn, to terminal behaviors. As evaluative 
data are obtained the network is rearranged to reflect confirmed relationships. 

ERIC 
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These relationships become a working document with which evaluator and 
developer begin revising and refining a program in order to bring about 
desired terminal behaviors. 
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Figure 6 Hypothesised Network of Constructs Before Formative 
Evaluation of a Teacher Training Program 
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